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I  Introduction 


This  is  a  continuation  of  the  previous  research  report,  which  is 
entitled,  "Information  Loss  Caused  by  Noise  in  Models  for  Dichotomous 
Items.”  In  the  present  paper,  we  will  focus  our  attention  upon  the  three- 
parameter  logistic  model,  which  is  widely  used  among  researchers  as  a 
model  for  the  multiple-choice  test  item  in  comparison  with  the  normal 
ogive  model. 

Throughout  this  paper,  we  shall  solely  consider  the  unidimensional 
latent  space.  Let  9  denote  the  latent  trait  which  assumes  any  real 
number.  In  dealing  with  the  multiple-choice  test  item,  there  exist  two 
distinct  standpoints:  1)  to  treat  the  item  as  a  dichotomous  item, 
classifying  the  correct  answer  into  one  category  and  all  the  other 
alternative  answers  into  the  other,  and  2)  to  treat  it  as  a  polychotomous 
item  by  acknowledging  each  alternative  as  an  individual  resource  of 
information.  In  the  former  case,  it  is  most  common  to  define  the  binary 
item  score,  u  ,  for  item  g  and  assign  u  *  1  to  the  correct  answer 

©  O 

and  Ug  *  0  to  all  the  other  alternative,  incorrect  answers.  If  we 
accept  the  knowledge  or  random  guessing  principle,  i.e.,  that  the  examinee 
either  knows  the  answer  or  guesses  randomly,  the  three-parameter  normal 
ogive,  or  logistic,  model  must  be  an  appropriate  model.  An  advantage  of 
the  model  may  be  its  simplicity.  Two  main  disadvantages  are,  however, 
that:  1)  in  many  practical  situations,  the  knowledge  or  random  guessing 
principle  is  not  applicable,  and  2)  because  of  the  noise  caused  by  random 
guessing,  we  must  do  without  certain  mathematical  properties  which 
otherwise  we  could  enjoy.  If  we  take  the  second  standpoint,  we  must 


estimate  the  operating  characteristic  of  each  of  the  incorrect  alternative 
answers,  which  sometimes  are  called  distractors,  in  addition  to  the  one 
for  the  correct  answer.  The  operating  characteristic  of  a  distractor  is 
called  the  plausibility  function.  A  family  of  models  for  the  multiple- 
choice  test  item,  which  takes  both  the  characteristics  of  each  distractor 
and  noise  caused  by  random  guessing,  has  been  proposed  (Samejima,  RR-79-4, 
Final  Report). 

In  comparison  with  the  first  standpoint,  there  is  no  question  that 
the  second  standpoint  is  better,  in  the  sense  that  each  test  item  will 
provide  us  with  a  greater  amount  of  information,  which  leads  to  the  more 
efficient  estimation  of  the  examinee's  ability,  or  latent  trait. 

Although  it  requires  more  mathematical  sophistication  in  dealing  with  it, 
it  may  be  time  that  researchers  switch  to  the  second  standpoint  and  enjoy 
its  benefits.  In  estimating  the  plausibility  functions  of  distractors, 
methods  and  approaches  for  estimating  the  operating  characteristics  of 
discrete  item  responses,  such  as  Levine's  (Levine,  1981)  and  Samejima's 
(Samejima,  1977a,  RR-77-1,  RR-78-1  to  RR-78-6,  RR-80-2,  RR-80-4,  RR-81-3, 
Final  Report),  will  be  useful.  It  has  been  found,  amazingly,  that  many 
existing  multiple-choice  test  items  have  informative  distractors.  In  the 
future,  however,  it  is  desirable  to  modify  the  guidelines  of  test 
construction  to  encourage  test  developers  to  include  more  informative 
distractors,  and  to  show  how  and  with  what  principle  they  should  do  that. 

At  present,  unfortunately,  this  second  tide  is  yet  to  come. 
Researchers  use  the  three-parameter  logistic  model  even  if  their  data 
contradict  the  knowledge  or  random  guessing  principle.  They  claim  that 


the  function  can  still  be  an  approximation  to  the  operating  characteristic 
of  the  correct  answer,  regardless  of  the  principle  it  may  follow. 

In  the  present  paper,  we  will  not  question  the  adequacy  of  the 
three-parameter  logistic  model  further  than  we  already  have.  We  will  put 
ourselves  in  an  assumption  that,  no  matter  what,  we  must  use  the  three- 
parameter  logistic  model  in  a  given  situation.  To  begin  with,  we  shall 
compare  the  three-parameter  logistic  model  with  the  normal  ogive  model  and 
find  out,  quantitatively,  how  much  the  three-parameter  logistic  model  has 
to  lose  and,  qualitatively,  what  kinds  of  deficiencies  the  model  has 
because  of  the  noise  caused  by  random  guessing. 

II  General  Characteristics  of  the  Three-Parameter  Logistic  Model 

Let  P  (9)  be  the  operating  characteristic  of  the  correct  answer  to 

o 

item  g  ,  or  the  item  characteristic  function.  This  function  is  the 
conditional  probability,  given  0  ,  with  which  the  subject  answers  item  g 
correctly.  Since  this  conditional  probability  al60  equals  the  mean  of  the 
conditional  distribution  of  the  binary  item  score  u  ,  given  0  ,  P  (0) 

o  o 

is  also  the  regression  of  the  binary  item  score  Ug  on  ability  0  . 
Three-parameter  logistic  model  is  defined  by  the  item  characteristic 
function  such  that 

(2.D  y9) "  cg  +  d-cg)  y9>  > 

where  Cg  is  a  constant  which  equals  the  reciprocal  of  the  number  of  the 
alternatives  attached  to  the  multiple-choice  test  item,  which  is  called 
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the  guessing  parameter,  and  f  (0)  is  given  by 

O 


(2.2) 


®g(0)  *=  [1  +  exp{-Dag(0-bg)}]-1 


This  function  v(0)  itself  is  the  item  characteristic  function  in  the 
© 

(two-parameter)  logistic  model.  The  two  item  parameters,  ag  and  bg  , 

in  (2.2)  are  called  the  item  discrimination  parameter  and  the  item 

difficulty  parameter,  respectively.  The  item  discrimination  parameter 

assumes  any  finite,  positive  value,  and  the  item  difficulty  parameter 

takes  on  any  finite,  real  number.  The  seemingly  redundant  constant,  D  , 

is  a  scaling  factor,  which  adjusts  the  value  of  the  discrimination 

parameter,  a  •  When  we  set  this  scaling  factor  equal  to  1.7  ,  the  same 
© 

set  of  two  item  parameters,  a  and  b  ,  provides  us  with  (0)  which 

o  ©  o 

is  very  close  to  the  corresponding  item  characteristic  function,  T  (0)  , 

© 

in  the  normal  ogive  model  (Birnbaum,  1968),  which  is  defined  by 


(2.3) 


*g(0)  -  (2*r1/2 


l  ag(9-V  e-„2/2 


du  . 


For  this  reason,  the  logistic  model  was  originally  developed  as  a 
substitute  for  the  normal  ogive  model.  The  former  provides  us  with  a 
sufficient  statistic,  t(V)  ,  which  is  given  by 


(2.4)  t(V)  -  I  a  u  , 

g-1  8  8 


for  the  response  pattern, 
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(2.5)  V  =  (uL,  U2*  •••  ,  Ug,  ...  ,  un) '  , 

where  n  is  the  number  of  items  in  the  test.  The  maximum  likelihood 

A 

estimate,  0y  .  of  ability  g  is  given  by  the  solution  of 

n 

(2.6)  t(v)  -  z  ag  ye) 

g=l 

for  the  (two-parameter)  logistic  model. 

It  has  been  reported  (e.g.,  Lord,  1968)  that  an  unrestricted 
estimation  of  the  three  parameters  in  the  three-parameter  logistic  model 
provides  us  with  the  estimated  guessing  parameter  which  is  substantially 
different  from  the  reciprocal  of  the  number  of  the  alternatives  attached 
to  the  multiple-choice  item,  and  very  often  the  value  Is  less.  Note  that 
this  fact  Itself  is  an  invalidation  of  the  model.  Many  researchers  stick 
to  the  model,  however,  as  a  simple  numerical  approximation  to  some  unknown 
item  characteristic  function,  which  exists  behind  their  empirical  data  and 
whose  formula  and  rationale  are  hidden.  For  this  reason,  this  third 
parameter  c  is  sometimes  called  pseudo-guessing  parameter.  This 

O 

interpretation  of  the  three-parameter  logistic  model  casts  some  doubt, 
however.  It  has  been  shown  (Samejiraa,  RR-79-4;  April  1980)  that,  if  our 
test  item  follows  a  model  in  the  new  family  of  models  for  the  multiple- 
choice  test  item,  the  operating  characteristic  of  the  correct  answer,  or 
item  characteristic  function,  tends  to  have  a  non-monotonic  form.  Figure 
2-1  presents  several  typical  item  characteristic  functions  in  Models  A,  B 


PROBABILITY  PROBABILITY 
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FIGLIRE  2-1 


Typical  Operating  Characteristics  of  the  Correct  Answer  in  Models 
A,  B  and  C.  Ability  Distribution  of  a  Group  of  Hypothetical 
Examinees  is  Also  Drawn  by  a  Thick,  Solid  Line. 
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and  C,  respectively,  which  were  taken  from  one  of  ;*•  previous  works 

(Samejima,  RR-79-4).  We  can  see  in  this  figure  that  each  operating 

characteristic  of  the  correct  answer  decreases  in  9  up  to  a  certain 

level,  and  then  starts  increasing.  If,  for  instance,  our  calibration 

data  have  been  collected  for  a  group  of  subjects  whose  ability 

distribution  follows  the  density  drawn  by  a  thick,  solid  line  in  each 

graph  of  Figure  2-1,  and  if,  nonetheless,  we  assume  the  three-parameter 

logistic  model  for  our  items,  then  the  estimates  of  the  parameters  cg 

will  be  less  than  the  reciprocal  of  the  number  of  the  alternatives.  For 

the  purpose  of  illustration.  Figure  2-2  presents  one  of  the  curves  of 

Model  A  with  a  =  1.00  ,  b.  -  -1.50  ,  -1.00  ,  -0.50  ,  0.00  ,  0.50  , 

8  g 

which  was  taken  from  the  first  graph  of  Figure  2-1,  by  a  solid  line,  and 
the  item  characteristic  function  in  the  three-parameter  logistic  model 
with  ag  *  1.00  ,  bg  =  0.60  and  a  pseudo-guessing  parameter  0.05  by  a 
dotted  line.  Comparison  of  these  two  curves  in  Figure  2-2  suggests  that, 
for  the  interval  of  0  where  most  of  the  subjects  of  our  calibration  data 
are  located,  these  two  item  characteristic  functions  are  practically 
identical.  And  yet  the  danger  of  accepting  the  three-parameter  logistic 
model  as  the  approximation  for  Model  A  is  obvious,  for  the  discrepancies 
are  substantial  outside  this  interval  of  0  .  If,  for  instance,  the 
estimated  item  characteristic  function  in  the  three-parameter  logistic 
model  thus  calibrated  is  applied  for  data  collected  for  another  group  of 
subjects  whose  ability  distribution  is  shifted  to  a  lower  side  of  9  , 
then  we  will  have  a  serious  problem  in  analyzing  our  data  because  of  these 
discrepancies.  This  fact  also  implies  the  danger  of  using  a  single  set  of 


4.0  -3.0  -2.0  -1.0  0.0  1.0  2.0  3.0  4.0 

LATENT  TRAIT  0 


FIGURE  2-2 

The  Item  Characteristic  Function  of  an  Item  Following  Model  A  with 
a  *  1 .00  ,  bx  =  -1.50  ,  -1.00  ,  -0,50  ,  0.00  and  0.50  for 
9  9 

(Solid  Line),  and  the  One  Following  the  Three- Parameter  Logistic 
a_  =  1.00  ,  b  =  0.60  and  c  =  0.05  (Dotted  Line) 


the  Parameters 
xg  =  1 ,2, 3, 4, 5 

Model  with 


data  in  model  validation.  It  should  be  kept  in  mind  especially  when  our 

model  fails  to  provide  us  with  a  sound  rationale,  as  the  three-parameter 

logistic  model  with  a  pseudo-guessing  parameter  does. 

It  has  been  pointed  out  (Samejima,  1972,  1973)  that,  unlike  the 

(two-parameter)  normal  ogive  and  logistic  models,  the  three-parameter 

logistic  model  does  not  provide  us  with  a  unique  modal  point  for  the 

likelihood  function  of  every  possible  response  pattern.  We  can  write  for 

the  basic  function  (Samejima,  1969,  1972),  A  (6)  ,  in  the  three- 

8 

parameter  logistic  model. 


(2.7) 


V9) 


{: 


-d*8vs) 


u, 


[(l-cg)  Dag  Yg(0){1"Ve)}]  tCg+a-Cg)^)] 


-1 


From  (2.7)  it  is  obvious  that  the  basic  function  is  not  strictly  decreasing 


in  8  for  u  ■  1  ,  although  it  is  for  u  ■  0  .  This  leads  to  the  fact 

o  © 


that,  while  either  in  the  normal  ogive  model  or  in  the  logistic  model  the 


item  response  Information  function  (Samejima,  1972),  I  (0)  ,  assumes 


positive  values  throughout  the  entire  range  of  ability  9  for  both  ug  ■  0 


and  u  *  1  ,  in  the  three-parameter  logistic  model  there  is  an 
8 


interval  of  9  where  I  (9)  assumes  negative  values  for  uff  *  1 

8  8 


This  interval  is  (-<*>,  9  )  ,  where  9  is  given  by 

O  © 


(2.8) 


(2Dag)_1  log  cg  +  bg 


Several  observations  were  made  for  the  item  response  information  function 
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in  the  three-parameter  logistic  model,  which  was  used  as  an  example  of  the 
Type  B  model  (Samejima,  ONR/RR-82-1) . 

Ill  Loss  of  Accuracy  in  Ability  Estimation  Caused  by  Random  Guessing 

In  this  section,  we  shall  observe  the  loss  of  accuracy  in 

estimating  the  examinee's  ability  caused  by  random  guessing,  by  comparing 

the  local  standard  errors  of  estimation  of  different  hypothetical  tests 

which  follow  the  normal  ogive  model  and  the  three-parameter  logistic  model 

with  c  =*  0.2  0  and  c  =  0.2  5  ,  respectively.  In  so  doing,  we  shall  use 
©  © 

hypothetical  tests  of  equivalent  items,  or  items  having  identical  item 
characteristic  functions.  It  was  pointed  out  in  the  preceding  section 
that  the  three-parameter  logistic  model  does  not  assure  a  unique  maximum 
for  the  likelihood  function  of  every  possible  response  pattern,  and  the 
item  response  information  function  for  Ug  -  1  assumes  negative  values 
for  the  interval,  (-  <*> ,  0g)  .  Thus  it  may  be  meaningless  to  discuss  the 
standard  error  of  estimation  when  single  maximum  likelihood  estimates  may 
not  exist  for  some  response  patterns.  If  a  test  consists  of  equivalent 
items,  however,  a  unique  maximum  likelihood  estimate  always  exists  for 
every  response  pattern,  regardless  of  the  model  the  items  follow.  In 
fact,  the  simple  test  score  t  ,  which  is  the  sum  total  of  the  binary  item 
scores,  is  a  simple  sufficient  statistic  for  the  response  pattern  V  ,  and 
the  maximum  likelihood  estimate,  8  ,  is  obtained  as  the  solution  of 

(t/n)  -  Pg(6)  * 


(3.1) 


Note,  however,  that,  if  the  item  follows  the  three-parameter  logistic 
model,  or  any  other  model  of  Type  B,  (3.1)  will  not  have  a  solution  if 


the  relative  test  score,  (t/n)  ,  is  less  than  c  .  In  such  a  case,  the 

O 

maximum  likelihood  estimate  is  negative  infinity  (cf.  Samejima,  ONR/RR-82-1). 

In  general,  for  a  binary  item  g  ,  the  item  response  information 
function,  Iu  (e)  ,  is  defined  by 


(3.2) 


V9)  * 


le?  log  Va) 

~  log  P  (e) 

362  8 


0 

1 


where  P  (e)  and  Q  (e)  [■  1  ”**„( 6)3  are  the  operating  characteristics 

O  o  © 

of  the  correct  and  incorrect  answers  for  item  g  ,  respectively.  In  the 

three-parameter  logistic  model,  P  (e)  is  given  by  (2.1)  with  (2.2) 

© 

substituting  for  ^  ( e )  ,  and,  in  the  normal  ogive  model,  it  is  replaced  by 

© 

the  right  hand  side  of  (2.3).  The  item  information  function,  I  (6)  ,  is 

© 

the  conditional  expectation  of  the  item  response  information  function, 

I  (0)  ,  given  e  ,  and  for  a  binary  item  g  we  obtain 
8 

(3.3)  ig(e)  -  E[I  (0)  |e]  -  1 PgC0)]2  [Pg(e)Qg(e)]_1  . 

.t  has  been  shown  (Samejima,  RR-79-1,  ONR/RR-82-1)  that  there  exists  some 
stancy  for  the  amount  of  information  given  by  a  binary  item,  and,  in 
particular,  models  of  Type  A  which  provide  us  with  strictly  increasing 
item  characteristic  functions  with  zero  and  unity  as  the  two  asymptotes, 
and  to  which  the  normal  ogive  model  belongs,  the  area  under  the  curve  of 


the  square  root  of  the  item  information  function  equals  ir  .  It  has  also 
been  pointed  out  (Samejima,  ONR/RR-82-1)  that,  for  models  of  Type  B, 
whose  item  characteristic  functions  are  given  by  (2.1),  and  to  which  the 
three- parameter  logistic  model  belongs,  this  area,  Q  ,  is  given  by 

(3.4)  Q  =*  it  -  2tan_1[cg/(l-cg)]1^2  . 

When  c  *  0.20  ,  Q  equals,  approximately,  0.705ir  ,  and  when  c  -  0.25 
®  8 

it  is  approximately  0.667  ir  . 

When  a  test  consists  of  only  one  item,  the  item  information 

function  I  (0)  equals  the  test  information  function  1(6)  ,  and  the 
© 

characteristics  of  the  square  root  of  the  item  information  function  apply 
directly  for  the  square  root  of  the  item  information  function.  In 
practice,  however,  it  is  a  highly  unlikely  case,  and  there  usually  are 
more  than  one  binary  item  in  a  test.  The  response  pattern  information 
function,  Iy(0)  »  is  defined  by 

(3.5)  Iy(9)  *  30^ log  Py(6)  , 

where  Py(0)  is  the  operating  characteristic,  or  the  conditional 

probability,  given  9  ,  of  the  response  pattern  V  .  When  the  conditional 

independence  of  the  item  score  distributions,  given  9  ,  holds,  this 

operating  characteristic  is  given  as  the  product  of  the  operating 

characteristics  of  u  which  belong  to  the  response  pattern  V  .  The 

© 

test  information  function  is  defined  as  the  conditional  expectation  of  the 
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response  pattern  Information  function,  given  9  ,  and  thus  we  can  write 

(3.6)  1(6)  -  E[Iv(0)  |0]  -  E  1(0)  Py(@)  . 

V 

From  (3.6),  following  through  some  mathematics,  we  obtain 

n 

(3.7)  1(9)  -  t  1(0)  , 

provided  that  the  conditional  Independence  of  the  item  score  distribution 
holds. 

We  notice  that,  if  the  term  under  the  summation  on  the  right  hand 
side  of  (3.7)  were  the  square  root  of  the  item  information  function. 
Instead  of  the  item  information  function  itself,  then  there  would  be  a 
similar  constancy  for  the  amount  of  test  information  as  there  is  for  the 
amount  of  item  information.  As  it  is,  however,  there  is  no  such  constancy 
for  the  square  root  of  the  test  information  function,  and,  therefore, 
different  combinations  of  items  will  provide  us  with  different  values  for 
the  area  under  the  curve  of  the  square  root  of  the  test  information 
function  . 

When  a  test  consists  of  n  equivalent  items,  however,  there  exists 
a  similar  kind  of  constancy  for  the  square  root  of  the  test  information 
function,  for  from  (3.7)  we  can  write 

(3.8)  [I(6)]1/2  -  n1/2[Ig(9)]1/2  , 
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where  I  ( 0)  Is  the  common  item  information  function.  This  indicates 

O 

that,  when  a  test  consists  of  n  equivalent  binary  items  following  the 

normal  ogive  model,  the  area  under  the  square  root  of  the  test  information 
1  /2 

function  equals  n  v  ,  regardless  of  the  common  item  parameters,  a 

O 

and  bg  ,  of  those  equivalent  items.  (3.8)  also  implies  that,  if  a  test 
has  n  equivalent  binary  items  following  the  three-parameter  logistic 
model  with  Cg  *•  0.2  0  ,  then  the  area  under  the  curve  of  the  square  root 
of  the  test  information  function  approximately  equals  0.705  n  '  tt  ,  and 
with  Cg  **  0.2  5  it  equals  approximately  0.667  n^^tr  ,  Figure  3-1 
illustrates  the  square  roots  of  test  information  functions  in  the  normal 
ogive  model  and  in  the  three-parameter  logistic  model  with  Cg  *  0.20  and 
c  *  0.2  5  with  the  other  common  parameters  a  =  0.50  and  b  «  0.00  , 

o  ©  © 

by  a  dashed  line  and  two  solid  lines,  respectively,  for  n  *  10,  20,  40, 


60,  80,  100,  120,  140,  160,  180,  200  .  A  similar  comparison  with  respect 
to  the  three  test  information  functions  was  made  and  is  shown  in  Appendix, 
as  Figure  A-l. 

We  notice  in  those  graphs  of  Figure  3-1  that  the  two  curves  for  the 

three-parameter  logistic  model  are  much  closer  to  each  other,  compared 

with  their  relationship  with  the  curve  for  the  normal  ogive  model.  The 

values  of  0  ,  which  were  obtained  by  (2.8),  turned  out  to  be 

© 

-0.9467282900048181  and  -0.8154673627399685  for  c  -  0.20  and 

g 

c  “  0.25  ,  respectively,  and  these  values  are  also  shown  in  Figure  3-1. 
© 

It  is  observed  that  the  distances  between  the  curve  for  the  normal  ogive 
model  and  each  of  the  two  curves  for  the  three-parameter  logistic  model 


are  rapidly  enhanced  as  0  departs  from  0  in  the  negative  direction. 

O 
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We  notice  that  those  graphs  of  three  curves  can  be  used  for  other 
sets  of  the  parameters  a  and  b  ,  as  long  as  we  keep  the  values  of  c 

5  6  © 

as  they  are.  In  so  doing,  we  need  to  change  the  scale  values  shown  or.  the 
abscissa  and  on  the  ordinate.  If,  for  instance,  we  wish  to  use  the  graphs 
for  a  =■  1.00  instead  of  a„  “  0.50  without  changing  the  value  of 

©  o 

bg  (*  0.00)  ,  then  we  must  change  the  numbers  on  the  abscissa  of  Figure 
3-1  to  their  half  values,  and  also  make  those  numbers  on  the  ordinate 
twice  as  large  as  the  original  values.  If  we  wish  to  use  Figure  A-l  in 
Appendix  for  the  same  purpose,  then  we  will  have  to  change  the  numbers  on 
the  abscissa  in  the  same  way  as  we  did  for  Figure  3-1  and  make  the  numbers 
on  the  ordinate  four  times  as  large  as  the  original  values. 

By  virtue  of  the  asymptotic  normality  of  the  conditional 
distribution  of  the  maximum  likelihood  estimate  \  ,  given  9  ( e.g ., 

Kendall  and  Stuart,  1961,  Samejima,  1975),  the  asymptotic  regression  of 
0V  on  0  equals  0  Itself,  i.e.,  the  maximum  likelihood  estimate  v 

is  asymptotically  conditionally  unbiased,  and  the  conditional  distribution 

1 12 

is  asymptotically  normal  with  9  and  [1(9)]  as  its  two  parameters. 

It  has  been  shown  (Samejima,  1977a,  1977b,  Final  Report)  that  even  for  a 
relatively  small  number  of  Items  and  a  relatively  small  amount  of  test 
information  this  asymptotic  property  can  be  used  as  a  good  approximation 
to  the  conditional  distribution  of  §v  .  Figure  3-2  presents  the  interval 
of  9y  which  was  obtained  by  using  9  as  the  regression  of  0y  on  9 
and  [1(9)] ^  ^  as  the  standard  error  of  estimation,  which  is  plotted  both 
above  and  below  the  regression  line.  In  this  figure,  the  two  curves  are 
drawn  by  solid  lines  for  the  normal  ogive  model,  and  they  are  drawn  by 
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dashed  and  dotted  lines  for  the  three-parameter  logistic  model  with 
c  =  0.20  and  c  =  0.25  ,  respectively,  for  each  of  the  ten  tests  of 

o  o 

equivalent  items.  The  number  of  items  in  those  ten  tests  are  20,  40,  60, 

80,  100,  120,  140,  160,  180  and  200  ,  respectively.  The  common  item 

parameters  are  the  same  as  those  used  in  the  eleven  tests  in  Figure  3-1, 

i  .  e  .  ,  a  ■=  0.50  and  b  =  0.00  .  On  both  the  abscissa  and  the  ordinate 
’  g  g 

of  Figure  3-2,  two  additional  sets  of  numbers  are  shown  in  the  smallest 
and  medium  sizes  to  adjust  the  scales  to  the  change  of  a^  from  0.50  to 
2.00  and  1.00  ,  respectively. 

We  can  see  in  these  ten  graphs  of  Figure  3-2  that  for  relatively 
large  numbers  of  items  the  three-parameter  logistic  model  provides  us  with 
reasonably  small  confidence  intervals  for  certain  intervals  of  0  ,  but 
the  accuracy  of  estimation  drops  radically  as  0  departs  from  6g  in  the 
negative  direction.  Although  this  tendency  is  conspicuous  for  smaller 
numbers  of  items,  it  is  still  clear  for  larger  numbers  of  items,  like 
200  .  The  effect  of  noise  caused  by  random  guessing  is  obvious,  and  we 
need  to  devise  some  way  to  make  up  for  it  if  we  must  use  the  three- 
parameter  logistic  model. 

IV  Loss  in  Speed  of  Convergence  of  the  Conditional  Distribution  of 
the  Maximum  Likelihood  Estimate  to  the  Normality 

In  the  preceding  section,  we  have  observed  the  loss  of  accuracy  in 

ability  estimation  caused  by  random  guessing,  which  is  embedded  in  the 

three-parameter  logistic  model.  In  so  doing,  we  compared  the  normal  ogive 

model  and  the  three-parameter  logistic  model  with  Cg  -  0.20  and 

c  ■  0.25  ,  with  respect  to  the  confidence  Interval  of  the  maximum 


likelihood  estimate  5  ,  given  0  ,  which  was  approximated  by  the 

unbiased  regression  and  the  reciprocal  of  the  square  root  of  the  test 
information  function  as  the  standard  error  of  estimation.  Those 
confidence  intervals  are  approximations,  using  the  asymptotic  normality  of 
the  maximum  likelihood  estimate,  given  9  . 

In  this  section,  we  shall  focus  our  attention  on  the  loss  in  the 
speed  of  convergence  of  the  conditional  distribution  of  the  maximum 
likelihood  estimate  0  to  the  normality,  which  is  caused  by  random 
guessing  embedded  in  the  three-parameter  logistic  model.  In  so  doing,  we 
shall  use  five  hypothetical  tests  of  equivalent  items,  i.e.,  n  =  20,  40, 
80,  120,  200  ,  which  were  chosen  out  of  the  ten  tests  used  in  the 
preceding  section  . 

We  notice  that,  with  equivalent  items,  the  conditional  distribution 
of  the  test  score  t  ,  given  9  ,  is  binomial,  with  n  and  Pg(0)  38  the 

two  parameters.  Since  there  is  one-to-one  correspondence  between  the  test 
score  t  and  the  maximum  likelihood  estimate  0t  ,  the  probability 

A 

function  of  t  also  applies  to  the  probability  function  of  0t  ,  if  each 
non-zero  probability  is  assigned  to  0  instead  of  t  .  Questions  are 
raised  as  to  how  close  this  discrete  distribution  is  to  N(9,  {1(6)}  , 

and  if  there  are  substantial  differences  between  the  two  models  with 
respect  to  the  speed  of  convergence  to  the  normality. 

We  notice  that  in  the  normal  ogive  model  ©t  assumes  negative  and 
positive  infinities  for  t  =  0  and  t  *  n  ,  respectively,  whereas  in  the 
three-parameter  logistic  model  it  does  for  t  <  nc  and  t  **  n  . 

O 

One  measure  of  the  speed  of  convergence  to  the  normality,  therefore,  is  the 
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probabillti.es  assigned  to  the  negative  and  positive  infinities  in  each 
distribution,  i.e.,  the  greater  these  probabilties  are,  the  slower  the 
convergence  to  the  normality  is.  Table  4-1  presents  these  probabilities 
assigned  to  the  negative  and  positive  infinities  at  each  of  the  sixteen 
equally  spaced  points  of  0  for  each  of  the  five  hypothetical  tests  of 
equivalent  items.  We  can  see  from  this  table  that,  in  all  five  situations, 
where  the  numbers  of  items  are  20,  40,  80,  120  and  200  ,  respectively, 
the  sum  totals  of  the  probabilities  assigned  to  the  negative  or  positive 
infinity  on  the  three-parameter  logistic  model  are  substantially  larger, 
compared  with  the  ones  on  the  normal  ogive  model. 

Since  the  probabilities  assigned  to  the  negative  and  positive 

a 

infinities  are  not  zero  in  each  conditional  distribution  of  0t  ,  given 
0  ,  the  moments  of  each  distribution  are  indeterminate.  As  a  crude 
measure,  however,  we  shall  compute  the  moments  of  each  distribution  by 
simply  "ignoring"  the  negative  and  positive  infinities  and  their 
corresponding  probabilities.  Tables  4-2  through  4-4  present  the  first 
moment  about  the  origin  and  the  econd  through  fourth  moments  about  the 
mean  thus  computed,  for  each  of  the  sixteen  values  of  0  .  For 
simplicity,  the  symbols  y^'  ,  and  are  used  for  those 

moments,  which  indicate 

(4.1)  u{  -  E(0t  !  9)  , 
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(4.2)  Uk  =  E( (9t-u{  }k I  0)  , 

for  k  =  2,3,4  .  In  Che  same  table,  also  presented  are  the  indices  8^ 
and  and  Pearson's  criterion  <  ,  which  are  defined  by 

(4.3)  B  =  y2  y-3  , 

1  3  2 

(4.4)  B2  -  y  y-2 

z  4  2 

and 

(4.5)  <  -  61(82+3)2[^(2B2-381-6)(482-381)]_1  . 

This  criterion  k  is  used  to  determine  which  type  of  Pearson’s 
distributions  should  be  chosen  to  fit  our  empirical  distribution  (cf. 

Elder ton  and  Johnson,  1969;  Johnson  and  Kotz,  1970),  and  those  types  are 
given  in  the  last  columns  of  Tables  4-2  through  4-4. 

We  realize  that  those  moments  and  indices  cannot  seriously  be  taken 
into  consideration  unless  the  two  probabilities  assigned  to  the  negative 
and  positive  infinities,  which  are  presented  in  Table  4-1,  are  negligibly 
small.  Those  cases  are  indicated  in  Tables  4-2  through  4-4  by  **  and  *  , 
the  former  of  which  indicates  that  the  sum  total  of  the  probabilities 
assigned  to  the  finite  values  of  9t  is  greater  than,  or  equal  to, 

0.999999  ,  and  the  latter  means  that  it  is  greater  than,  or  equal  to, 


0.99  but  less  than  0.999999  . 
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If  a  distribution  is  normal,  then  we  will  have  =  0.00  ,  62  *■  3.00 

and  the  criterion  <  converges  to  zero.  We  anticipate  that,  since  those 

five  hypothetical  tests  consist  of  equivalent  items,  in  the  normal  ogive 

model  the  convergence  to  the  normality  must  be  speediest  at  0  =  b  ,  and 

S 

in  the  three-parameter  logistic  model  it  must  be  fastest  at 


(A. 6) 


0  -  b  +  (Da  r1  log[ {l~4c  +(l+8c  )1/2}{3-(l+8c  )L^2 }_1]  , 


at  which  the  common  item  information  function  I  (0)  assumes  the  highest 

8 

value.  With  our  hypothetical  tests  of  equivalent  items,  those  values  are 
0.00  for  the  normal  ogive  model,  0.31428  for  the  three-parameter 
logistic  model  with  c  =  0.20  ,  and  0.36695  for  the  three-parameter 

O 

logistic  model  with  Cg  -  0.25  . 

Comparison  of  those  three  tables  indicates  that  there  exist 
substantial  differences  in  the  convergence  to  the  normality  of  the 
conditional  distribution  of  0t  ,  given  0  ,  between  the  results  on  the 
normal  ogive  model  and  those  on  the  three-parameter  logistic  model.  For 
example,  for  n  =  200  ,  the  result  on  the  normal  ogive  model  provides  us 
with  the  regression  which  differs  from  0  by  less  than  0.01  in  absolute 
value,  B^  which  is  less  than  0.01  ,  which  differs  from  3.00  by 

less  than  0.10  in  absolute  value,  and  k  which  is  less  than  0.10  in 

absolute  value,  for  as  many  as  six  points  of  0  ,  i.e.,  -1.0  ,  -0.6  , 

-0.2  ,  0.2  ,  0.6  and  1.0  ,  whereas  the  same  is  true  for  the  results  on 

the  three-parameter  logistic  model  only  for  three  points  of  0  ,  i.e., 

-0.2  ,0.2  and  0.6  in  each  of  the  two  cases  where  c^  ■ 


0.20  and 
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c  =  0.25  ,  respectively.  As  another  example,  for  n  =  80  ,  the  above 

is  true  for  as  many  as  four  points  of  9  ,  i.e.,  -0.6  ,  -0.2  ,  0.2  and 

0.6  ,  on  the  normal  ogive  model,  while  at  no  point  of  9  is  it  satisfied 

on  the  three-parameter  logistic  model  in  either  of  the  two  cases  where 

c  =  0.20  and  c  =  0.25  .  If  we  relax  this  rule  to  !p.'-0|<O.O3  , 
g  g  T- 

8^  <  0.05  ,  |  |  <  0*50  ,  and  |<t  <  0.25  ,  then  the  resultant  number  of 

points  at  which  this  condition  is  satisfied  is  as  shown  in  Table  4-5  for 
each  test  and  each  model.  From  this  table,  too,  we  can  see  an  obvious 
effect  of  noise  caused  by  random  guessing  in  the  three-parameter  logistic 
model  on  the  speed  of  convergence  of  the  conditional  distribution  of  6  , 

given  0  ,  to  the  normality. 

Figures  4-1  through  4-3  present  ,  that  is  the  regression  of  0t 
on  9  ,  and  the  confidence  interval  »  p{+P2^)  plotted  against 

the  sixteen  points  of  0  ,  together  with  the  asymptotic,  unbiased 
regression  with  the  confidence  interval  (0 -{ 1(0)  }-*^,  0+{I(0)}  , 

which  were  first  presented  in  Figure  3-2  of  the  preceding  section.  Since 
those  moments  were  computed  by  "ignoring”  negative  and  positive 
infinities,  we  cannot  take  some  sets  of  three  points  seriously  if  the 
probabilities  assigned  to  either  negative  or  positive  infinities,  or  both, 
are  substantially  large.  For  this  reason,  in  Figures  4-1  through  4-3,  five 
different  symbols  are  used  to  indicate  the  magnitude  of  the  sum  total  of 
the  probabilities  assigned  to  the  finite  values  of  0  ,  which  are  shown 

in  Table  4-6.  The  S  shape  observed  in  some  of  the  graphs  in  those 
three  figures  is  caused  by  smaller  values  of  the  sum  total  of  those 
probabilities  .  In  those  graphs,  when  this  sum  is  greater  than,  or  equal 
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TABLE  4-5 


Number  of  Points  of  0  at  which  |pj-e|  <  0.03  , 

S-|<  0.05  ,  1 82~3 1  <  0.50  and  <  <  0.25  ,  for 

Each  of  the  Five  Tests  of  Equivalent  Items 
Following  the  Normal  Ogive  Model  and  the 
Three- Parameter  Logistic  Model  with 
c  *  0.20  and  c  =  0.25  . 

9  9 
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(0-0(9)}  ^  ,  9+0(0)}  ^  )  ,  for  Each  of  the  Five  Tests  of  Equivalent  Items  on  the 
Three-Parameter  Logistic  Model  with  a  =  0.50  ,  b  -  0.00  and  c  =  0.20  . 
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TABLE  4-6 

Symbols  for  the  Five  Different  Categories  of 
the  Sum  of  the  Probabilities  Assigned  to 
Finite  Values  of  the  Maximum  Likelihood 
Estimates . 


Lower 
Endpoint 
( inclusive) 

Upper 

Endpoint 

(exclusive) 

Mark 
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0.80 

■ 

0.80 

0.90 
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0.90 

0.95 
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0.95 

0.99 
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-•< 

0.99 
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to,  0.999999  for  two  or  more  adjacent  points  of  5  ,  the  solid  circles  are 
connected  by  a  thicker  line. 

We  can  see  from  those  three  figures  that  those  regressions  and 
confidence  intervals  are  very  close  to  the  asymptotic  ones,  for  the  points 
of  3  at  which  the  sum  total  of  the  probabilities  for  finite  values  of 
2t  is  no  less  than  0.99  and,  especially,  when  the  sum  is  greater 
than,  or  equal  to,  0.999999  ,  they  practically  lie  on  those  lines. 

Again  ,  we  can  see  substantial  differences  with  respect  to  the  agreement 
between  the  results  on  the  normal  ogive  model  and  those  on  the  three- 
parameter  logistic  model. 

V  Comparison  of  Tests  of  Non-Equivalent  Items 

We  have  seen  in  the  preceding  section  that  the  speed  of  convergence 

to  the  normality  of  the  conditional  distribution  of  the  maximum  likelihood 

estimate,  given  0  ,  is  fairly  high,  even  for  a  small  number  of  equivalent 

items,  if  the  values  of  &  are  close  to  the  point  at  which  the  amount  of 

test  information  is  maximal,  in  each  of  the  three  cases  In  which  the  normal 

ogive  model  and  the  three-parameter  logistic  model  with  c  ■  0.20  and 

© 

c  =  0.25  are  followed.  The  range  of  0  for  which  this  is  the  case  is 
fairly  small,  however,  for  a  smaller  number  of  equivalent  items,  especially 
on  the  three-parameter  logistic  model.  We  notice  that  this  interval  of  0 
may  be  enhanced,  if  we  use  non-equivalent  test  items.  For  this  reason, 
in  the  present  section,  we  shall  observe  the  regressions  and  confidence 
intervals  of  three  hypothetical  tests  of  ten  non-equivalent  items,  which 
follow  the  normal  ogive  model  with  the  common  discrimination  parameter 
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a  =  0.50  and  with  the  difficulty  parameters  b  shown  in  Table  5-1 

o  8 


for  the  ten  items  of  each  of  the  three  tests,  in  comparison  with  those  of 
a  hypothetical  test  of  ten  equivalent  items  which  follow  the  same  model 


with  =  0.50  and  =  0.00  .  For  convenience,  hereafter,  we  shall 


call  any  tests  having  the  sets  of  difficulty  parameters  shown  in  Table  5-1, 
Cases  1,  2  and  3,  respectively. 

As  we  did  for  the  five  hypothetical  tests  of  20,  40,  80,  120  and 
200  equivalent  items,  the  results  of  which  were  shown  in  the  preceding 
section,  the  four  conditional  moments,  p ,  P2  >  ^3  and  U4  >  were 
computed  by  "ignoring"  the  probabilities  assigned  to  the  negative  and 
positive  infinities,  for  the  hypothetical  test  of  ten  equivalent  items  and 
for  the  three  hypothetical  tests  of  ten  non-equivalent  items.  Table  5-2 
presents  those  probabilities  assigned  to  the  negative  and  positive 
infinities  and  their  sum  totals,  for  the  four  hypothetical  tests  of  ten 
items.  Unlike  for  the  tests  of  equivalent  items,  for  the  three 
hypothetical  tests  of  non-equivalent  items  the  test  score  t  is  not  a 
simple  sufficient  statistic  for  the  response  pattern  V  ,  so  the  maximum 


likelihood  estimate  9y  must  be  obtained  as  the  solution  of 


(5.1) 


~  Au  (0)  =  0  , 

u  cV  8 
g 


where  A  (')  is  the  basic  function  (Samejima,  1969,  1972)  defined  by 


(  .  2  ) 


A  (  -,) 

V 


log  Qg(-) 


log  lye) 


«p.  =  1 


•4 


•  - 
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TABLE  5-1 

Difficulty  Parameter  b  of  Each  of  the 

9 

Ten  Items  in  Each  of  the  Three  Cases. 
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Case  1 

Case  2 

Case  3 
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with  the  item  characteristic  function  P  (9)  replaced  by  the  formula  for 

o 

that  of  the  normal  ogive  model,  which  is  given  on  the  right  hand  side  of 

(2.3) ,  and  Q  (6 )  =  1  -  P  ( 0)  .  The  conditional  distribution  of  the 

o  o 

maximum  likelihood  estimate  9y  ,  given  9  »  cannot  be  simplified  to  any 
form  as  we  did  for  equivalent  items  using  the  probability  function  of  the 
binomial  distribution  of  the  test  score,  for  the  recoonse  pattern  V  is 
not  unidimensionally  ordered,  and  for  other  reasons.  Thus  in  obtaining 
moments  we  must  use  the  operating  characteristic  Py(S)  itself  as  the 
probability  assigned  to  the  maximum  likelihood  estimate  9y  ,  which  is 
obtained  from  the  operating  characteristics  of  the  item  score  through  the 
formula 

(5.3)  pv(6)  -  t  p  (e)“8  QgCe)1'”8  . 

Vv 

provided  that  the  distributions  of  the  item  score  u^  are  conditionally 
independent,  given  9  ,  for  all  the  items  of  the  test.  Thus  the 
computation  of  the  moments  is  more  complicated  and  time  consuming,  and  we 
must  deal  with  as  many  as  2n  different  values  of  6y  instead  of  (n+1) 
different  values  of  9  .  For  this  reason,  we  are  forced  to  restrict  our 
observations  to  tests  of  only  ten  items,  in  which  the  number  of  different 
values  of  Gy  is  1,024  . 

Table  5-3  presents  the  four  moments  and  indices  f ^  and  ^  »  an^ 
Pearson's  criterion  k  and  distribution  type,  for  twenty  equally  spaced 
values  cf  0  ,  -3.0  through  3.0  ,  for  each  of  the  four  tests  of  ten 
items-  lust  as  in  Tables  4-2  through  4-4  ,  the  symbol  **  indicates  that 
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the  sum  total  of  the  probabilities  assigned  to  the  finite  values  of  Qy 
is  greater  than,  or  equal  to,  0.999999  ,  and  *  means  that  it  is  greater 
than,  or  equal  to,  0.99  ,  but  less  than  0.999999  .  We  can  see  that  none 
of  the  rows  of  the  table  are  marked  with  **  ,  unlike  the  other  cases  in 
which  the  number  of  items  is  20  or  greater,  which  we  observed  in  the 
preceding  section.  Comparison  of  the  results  of  the  three  non-equivalent 
cases  with  those  of  the  equivalent  case  reveals  that,  around  9  =  0.00  , 
which  equals  the  common  difficulty  parameter  b  for  the  equivalent  case 

o 

and  the  mean  of  the  difficulty  parameters  for  each  of  the  three  non¬ 
equivalent  cases,  the  conditional  distribution  of  6y  ,  given  5  ,  is 
closer  to  the  normality  for  a  wider  range  of  0  ,  as  the  difficulty 
parameters  spread  more  widely.  If  we  take  the  arbitrary  criterion  as  we 
did  in  the  preceding  section,  i.e.,  jy^-0  [  <  0.03  ,  <  0.05  , 

I  e2-3  l  <  0.50  and  I K  !  <  0.25  ,  then  this  criterion  is  satisfied  for  the 
six  values  of  9  ,  -1.0  ,  -0.6  ,  -0.2  ,  0.2  ,  0.6  and  1.0  ,  for  Case  3, 
while  this  number  reduces  to  four  in  Case  2,  and  two  for  Case  1  and  for 
the  equivalent  case. 

Figure  5-1  presents  the  two  kinds  of  regression  plus  confidence 
interval  for  each  of  the  four  cases,  which  were  observed  in  the  preceding 
section  for  each  of  the  five  equivalent  tests  on  the  normal  ogive  and  on  the 
three-parameter  logistic  models.  The  two  additional  sets  of  numbers  shown 
on  both  the  abscissa  and  the  ordinate  are  the  same  scale  changes  for 
a  =  2.00  and  a  =  1.00  ,  that  are  shown  in  Figures  4-2  through  4-4  . 

o  ? 

Note,  however,  following  each  scale  change,  the  difficulty  parameters  in 


the  three  non-equivalent  cases  are  also  changed  proportionally.  We  can 


10  Non- Equivalent  Items 
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see  that  in  each  case,  those  solid  circles,  which  are  based  upon  the  sum 
total  of  the  probabilities  assigned  to  the  finite  values  of  By  greater 

than,  or  equal  to,  0.99  ,  practically  lie  on  the  asymptotic  regression 

and  the  confidence  interval,  i.e.,  0  and  (0- {!( 9)  }-^^,  0  +  {I(0)}-^^)  . 

There  are  substantial  differences  in  the  range  of  0  for  which  this  is 
the  case,  however.  This  interval  of  6  is  (-2.2,  2.2)  for  Case  3 
of  the  non-equivalent  items,  while  it  is  only  (-0.6  ,0.6)  for  the 
equivalent  case. 

The  square  root  of  the  test  information  function  of  each  of  the 

three  non-equivalent  cases  is  shown  by  a  dashed  line  in  each  graph,  in 

both  Figures  5-2  and  5-3.  In  the  same  figures,  also  presented  by  solid 

lines  are  the  square  roots  of  the  test  information  functions  of  the  ten 

item  tests  which  are  based  upon  the  three-parameter  logistic  model  with 

the  same  parameters  ag  and  bg  and  the  third  parameter,  cg  ■  0.20  and 

cg  =  0.25  ,  respectively.  The  critical  values,  0g  ,  of  the  first  items 

of  the  three  tests  are  -0.3646728290004818D  01  ,  -0 . 4  5467  28290004819D  01 

and  -0 . 54467 28290004819D  01  for  c  **  0.20  ,  and  -0 . 3  515467362739969D  01  , 

© 

-0 .4 415467 3627 39969D  01  and  -0.5315467362739969D  01  for  cg  -  0.25  . 

In  Case  1,  the  values  of  0  increase  by  0.6  as  the  value  of  the 
difficulty  parameter  increases,  while  in  Cases  2  and  3  the  steps  are  0.8 
and  1.0  ,  respectively. 

If  we  take  the  strategy  not  to  use  the  informaton  obtained  from 
item  g  for  the  interval  of  9  ,  (-  00 ,  0  )  ,  then  the  square  root  of  the 
test  information  function  will  be  reduced  to  the  curve  plotted  by  a  dotted  line 
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avoid  multi-modal  likelihood  functions  which  may  occur  for  some  response 
patterns  if  the  items  follow  the  three-parameter  logistic  model.  Further 
investigation  as  to  the  merit  and  demerit  of  this  strategy  is  yet  to  come, 
however,  and  its  results  are  left  to  another  paper.  The  corresponding  six 
graphs  of  the  test  information  functions  are  presented  as  Figures  A-2  and 
A- 3  in  Appendix. 

Vi  Discussion  and  Conclusions 

The  three-parameter  logistic  model  was  compared  with  the  normal 
ogive  model  using  hypothetical  tests  of  equivalent  items,  mainly  with 
respect  to  the  speed  of  convergence  of  the  conditional  distribution  of 
the  maximum  likelihood  estimate,  given  0  ,  to  the  normality,  and  it 
was  found  out  that  the  effect  of  noise  caused  by  random  guessing  is 
substantial,  especially  for  the  values  of  0  less  than  the  critical  value 
.  Some  observations  were  made  on  the  normal  ogive  model  as  to  how 
the  interval  of  0  for  which  the  approximation  of  the  conditional 
distribution  by  the  normality  is  acceptable  can  be  enhanced  by  using  non- 
eoulvalent  test:  items. 

This  is  jus!  a  beginning  of  the  investigation  about  how  and  in 
what  ways  we  can  amend  the  deficiencies  of  the  three-parameter  logistic 
model  which  are  caused  by  random  guessing.  The  effective  use  of  the 
critical  value  may  he  one  solution  to  the  problem,  which  will  be 

investigated  further . 
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